The present invention relates to electromagnetic imaging, more particularly to methods and devices involving fusion of images from plural electromagnetic spectral band regions.
Traditional approaches to navigating waters or detecting threats under low-light conditions have involved radar systems, infrared (IR) devices, or electro-optic/light-intensification (EO/LI) devices. Radar devices used for navigation/detection purposes are usually “active” in the sense that they transmit electromagnetic radiation—more specifically, radio signals—in order to receive reflected radio signals that are informative as to existence of targets (objects) and their whereabouts. IR devices and EO/LI devices used for such purposes are “passive” in the sense that they do not transmit electromagnetic radiation, but instead receive naturally occurring electromagnetic signals that are emanated or reflected by targets.
In the electromagnetic spectrum, the IR region is characterized by longer wavelengths than is the visual region, the IR region extending between the visual region and approximately one millimeter in wavelength. The millimeter wave region is characterized by longer wavelength than is the IR region, and is characterized by shorter wavelengths than is the “radar” (including microwaves and radio waves) region. Passive EO/LI devices operate in the visible spectrum. Passive IR devices are based on the phenomenon of natural radiation of IR energy by all “warm” objects in accordance with thermal radiative transfer physics. Passive millimeter wave (mmW) devices have more recently come under development and are similar in principle to passive IR devices, a major difference being that mmW energy emissions by objects are at longer wavelengths than are IR emissions.
Operational effectiveness of passive (non-transmitting) imaging devices can be severely inhibited by atmospheric obscurants (e.g., particulate matter) characterizing inclement weather or air pollution. Obscurants can be precipitational (e.g., wet particulates) or non-precipitational (e.g., dry particulates). Examples of atmospheric obscurants include mist, fog, haze, dust, smoke, rain, drizzle (light rain), snow, snow grains, snow pellets, ice pellets (sleet), hail, etc.
Every kind of imaging device has its attributes and shortcomings. Passive mmW devices are advantageous over passive IR devices and passive EO/LI devices insofar as propagation of mmW energy is unaffected (or nearly unaffected) by environmental conditions that scatter or otherwise impede propagation of IR energy and visible light. Active radar devices share with passive mmW devices the quality of not suffering significant obscuration during adverse environmental propagative conditions.
Visual and IR devices are impaired by adverse conditions (e.g., inclement weather) and are thus ineffective for navigational systems under such conditions. Of all of the imaging devices, visual light devices can afford the highest image detail; on the other hand, visual spectrum systems operate during daytime only, are impaired by adverse weather/particulates, and are limited in range by the visual acuity of the device and the size of the target.
Similarly, IR devices provide good image detail, can be impaired by adverse weather/particulates, and are limited in range by the visual acuity of the device and the size of the target; however, as distinguished from visual devices, IR devices can operate in either day or night. Like IR devices, mmW devices can operate in either day or night; unlike IR devices, mmW devices afford marginal image detail, can operate in all weather/particulate conditions, and are limited to short range use.
Radar devices can operate in day or night and in all weather/particulate conditions. In addition, radar can yield range, speed, and course data, over time. However, radar provides no image detail, and active radar is limited to line-of-sight sensing. Furthermore, an active radar device requires a quantity of radar energy to be transmitted therefrom in order to illuminate objects such as navigational obstacles. This actively emitted radar energy can be collected by other sensors, and thus can be counterproductive to concealment of the entity (e.g., ship or other vehicle) from which the radar is being transmitted.
Generally speaking, the notion is well appreciated in the literature that image data from plural electromagnetic spectral band regions can be combined (“fused”) so as to take advantage of the respective sensory strong suits in the different spectral band regions. The following two textbooks, incorporated herein by reference, are informative about image fusion, i.e., the combination of image data from plural spectral band regions: Rick S. Blum and Zheng Uu, Multi-Sensor Image Fusion and Its Applications, CRC Press, 2005; Tania Stathaki, Image Fusion: Algorithms and Applications, Academic Press, 2008.
Also notable regarding image fusion are the following United States patent documents, each of which is incorporated herein by reference: Catano et al. U.S. Pat. No. 4,086,616 issued 25 Apr. 1978; Fitzpatrick et al. U.S. Pat. No. 5,072,396 issued 10 Dec. 1991; Smith U.S. Pat. No. 6,621,764 B1 issued 16 Sep. 2003; Appenrodt et al. U.S. Pat. No. 6,597,984 B2 issued 22 Jul. 2003; Chethik U.S. Pat. No. 6,816,112 B1 issued 9 Nov. 2004; Evans et al. U.S. Pat. No. 6,882,409 B1; Frady et al. U.S. Pat. No. 6,903,676 B1 issued 7 Jun. 2005; Chen et al. U.S. Pat. No. 6,909,997 B2 issued 21 Jun. 2005; Cook U.S. Pat. No. 6,919,988 B2 issued 19 Jul. 2005; Chen et al. U.S. Pat. No. 6,944,566 B2 issued 13 Sep. 2005; Frady et al. U.S. Pat. No. 7,049,998 B1 issued 23 May 2006; Connors et al. U.S. Pat. No. 7,053,928 B1 issued 30 May 2006; Lovberg et al. U.S. Pat. No. 7,170,442 B2 issued 30 Jan. 2007; Betush et al. U.S. Pat. No. 7,290,710 B2 issued 6 Nov. 2007; Abernathy U.S. Pat. No. 7,298,869 B1 issued 20 Nov. 2007; Giakos U.S. Pat. No. 7,420,675 B2 issued 2 Sep. 2008; Roy et al. U.S. Patent Application Publication 2008/0215204 A1 published 4 Sep. 2008; Giakos U.S. Pat. No. 7,428,050 B2 issued 23 Sep. 2008; Robertson et al. U.S. Pat. No. 7,439,902 B2 issued 21 Oct. 2008; Fox et al. U.S. Patent Application Publication 2009/0015460 A1 published 15 Jan. 2009.